As techniques to support various actions by using a video image where a CG image is superimposed on a video image of the real world are actively studied as Augmented Reality (AR) or Mixed Reality (MR) techniques. For example, the purposes of medical support by visualization of the inside of a patient body, work assistance by overlay-displaying a product assembly procedure on real parts in a factory, further, guidance by displaying place names and landmark names on a real landscape, and the like are known.
In CG image superimposition, some real-space area is hidden by a CG image. There is no problem if the user has no interest in the covered area. However, conventionally, even a real-space area to which the user pays attention could be hidden by the CG image since CG image superimposition is made just according to predetermined settings, without considering whether the area to be hidden is of interest or not.
G. Reitmayr and D. Schmalstieg show in “Mobile Collaborative Augmented Reality” In Proc. IEEE Virtual Reality 2001, pages 114-123, study of appropriate CG image display position particularly in a mobile AR system from analysis or CG image display position (Head-stabilized: information is displayed in a fixed position in correspondence with a user's view point; Body-stabilized: information is displayed in a fixed position in correspondence with the position of the user's body; and World-stabilized: a display position of information is controlled in correspondence with a position in the real world). Further, Japanese Published Unexamined Patent Application No. 2000-276610 (JP-A-2000-276610) relating to agent display or the like, discloses a system to display a user-friendly agent character. However, in both systems, some heuristics are employed as to CG image display position, and there is no consideration about the case where a target real space area is hidden by a CG image.
Further, a technique to display a particular real object (e.g., a user's hand) as an always-visible object (mask processing technique) is realized by using a chroma key technique or the like. However, the real world changes every second, and the user does not always pay attention to the same real object.
From this situation, generally, in a case where a real-space area to be observed by the user is hidden by a CG image, (1) the user removes an HMD, or if possible, (2) the CG image and video image (images of real world) are selectively displayed, or further, if possible, (3) a virtual object (forming the obstructive CG image) is selected and moved to another position. In the case of (2), because the whole CG image is displayed/not displayed at once, a part of the CG image cannot be selectively displayed. Further, in the case of (3), the operation of “selecting and moving” the virtual object is troublesome.